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OBJECTIVES OF THE PENNSYLVANIA RETRIEVAL OF INFORMATION 
FOR MATHEMATICS EDUCATION SYSTEM (PRIMES) PROJECT WERE TO 
DEVELOP AN INFORMATION STORAGE AND RETRIEVAL SYSTEM FOR 
ELEMENTARY SCHOOL MATHEMATICS CURRICULUM MATERIALS AND TO 
ESTABLISH PROCEDURES FOR LOCAL PENNSYLVANIA SCHOOL DISTRICTS 
TO USE THE SYSTEM FOR CURRICULUM DEVELOPMENT AND SELECTING 
INSTRUCTIONAL MATERIALS. BECAUSE BRIEF EXPERIENCE WITH THE 
SYSTEM PRECLUDES DISCUSSING EFFECTIVENESS, THIS REPORT 
DESCRIBES THE DEVELOPMENT AND USE OF PRODUCTS. A CENTRAL 
MICROFILM DOCUMENT FILE, CONSISTING OF 25.000 ANALYZED PAGES 
FROM ELEMENTARY MATHEMATICS BASAL TEXT SERIES, AND 
ACCOMPANYING TOOLS, INCLUDING CONTENT AND BEHAVIOR AUTHORITY 
LISTS, WERE DEVELOPED BY SUBJECT SPECIALISTS AND PREPARED BY 
COMPUTER. TWO REGIONAL CENTERS WERE ESTABLISHED FOR SERVICE ’ 
TO LOCAL SCHOOLS, AND FRIMES PERSONNEL HAVE PARTICIPATED IN 
TEACHER EDUCATION PROGRAMS. EDUCATIONAL CONSULTANTS WHO 
STUDIED THE OPERATION RECOMMEND THAT THE SYSTEM BE APPLIED 
AND DATA COLLECTED FOR EVALUATION. CONCLUSIONS ARE THAT (1) 
ANALYSIS OF MATERIALS CAN BE BONE CENTRALLY BY A STATE 
EDUCATION AGENCY, (2) IT IS FEASIBLE TO APPLY NEW INFORMATION 
TECHNOLOGY TO EDUCATIONAL CURRICULUM MATERIALS, AND (3) 

SERVICE TO LOCAL SCHOOL DISTRICTS SHOULD BE DECENTRALIZED. 
EXPANDING THIS SYSTEM AND MOVING TO OTHER AREAS AND 
STATE-WIDE SERVICES ARE SUGGESTED. APPENDIXES LIST PROJECT 
PERSONNEL, BASAL TEXTBOOK PROGRAMS IN FILE, AND FILE 
PARAMETERS. (JB) 
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INTRODUCTION 



"The Development of an Information System for Elementary 
School Mathematics Curriculum Materials: A Feasibility 

Study," Project No. 6-8152, was partially supported by 
funds under a Small Projects Proposal submitted to the 
U. S. Office of Education, Bureau of Research, September 
14, 1965, Contract No. QE-6-10-352. The grant was 
awarded to the Department of Public Instruction, Common- 
wealth of Pennsylvania, to assist in applying the latest 
technology of indexing, microfilming, storage, and re- 
trieval to a file of analyzed curriculum materials and 
to assess the feasibility of using this file with local 
school districts. The development of the file of infor- 
mation was initiated by Doris E. Creswell, Elementary 
Mathematics Adviser, and Emanuel Berger, Educational Re- 
search Associate, on January 1, 1965. Additional support 
was provided by the Commonwealth of Pennsylvania, the 
National Defense Education Act-Title III, and the Ele- 
mentary and Secondary Education Act-Titles III and V. 



THE PROBLEM 

Every school district in the Commonwealth of Pennsylvania 
is faced periodically with the need to select an elementary mathe- 
matics text series. This decision usually commits a district to the 
use of a particular text series for several years and exerts a 
strong influence on the development of the professional staff. 

These factors require that a careful selection be made. This in- 
volves an analytical stud/ of instructional objectives and the 
construction of a curriculum guide that reflects these objectives. 

A consideration of the role of mathematics instruction in the total 
curriculum, the qualifications of teachers, and the needs of the 
learners should precede these activities. Textbook selection 
criteria would have to be developed, the sanple books would have to 
be acquired, and a cornprehens ive survey of each one would be neces- 
sary. Most school districts have neither the time nor the resources 
to develop curriculum guides and select appropriate instructional 
materials . 

Thus, the selection task confronting each local school 
district is enormous. Over 30,000 pages of material should be read 
just to examine the current basal programs. Prior to mathematical 
and pedagogical assessment, a set of reliable criteria must be de- 
veloped for curriculum guide construction and textbook selection. 
Devising such criteria, alone, usually convinces a conscientious 
administrator that the task exceeds local resources. Frequently, 
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the available criteria are stated in terms too general for effective 
program conparison. For example, one criterion published recently 
by a leading state education agency states: 

The series of texts shall provide for content 
for each grade which is adapted to the needs, 
interests, and abilities of pupils. 

To apply this guideline, a local cannittee must develop a 
list of "needs, interests, and abilities of pupils... for each grade 
..." It would then have to examine each program and determine 
whether or not it satisfies the criterion. The difficulty of the 
task is further compounded if a district attempts to evaluate 15-20 
criteria items, many as comprehensive as the example cited. If 
these tasks were repeated in each of the Commonwealth * s 500 school 
districts, they would indeed represent an extraordinary, if not pro- 
hibitive, investment in man-days and effort. 

The technology for analyzing, storing, retrieving, and 
disseminating large volumes of information has been developed. For 
instance, the Smithsonian Institute is operating a Science Informa- 
tion Exchange that "... has information about over 40,000 biomedical 
research projects ..." Coverage is being extended to the physical 
and social sciences. Since considerable experience has been ac- 
cumulated in working with information problems in other disciplines, 
it is reasonable to assume that this knowledge will be applicable to 
curriculum materials; especially since neither the volume of pub- 
lished curriculum materials nor their complexity are as great as the 
problems encountered when working with scientific information. 

The Pennsylvania Retrieval of Information for Mathematics 
Education System, hereafter referred to as PRIMES, has utilized this 
modem technology to process the information in twenty published 
elementary textbook series. A center for this information has been 
established at the Department of Public Instruction. Each school 
district in the Commonwealth has access to the information for cur- 
riculum decision-making. PRIMES is based on the policy that mathe- 
matics curriculum decisions are the responsibility of the local 
school districts. The System is not intended to recommend specific 
basal programs for use in the Commonwealth . The purpose of the 
System is to provide school districts with an auxiliary tool for 
developing and selecting instructional materials consistent with 
their own educational aims. 



RELATED LITERATURE 



Research and development activity in "information storage 
and retrieval" is comparatively recent. Most studies in this 
field have been published since 1960. Nevertheless, in this short 
time, one bibliography cites almost 100 references dealing with 
applications to social science, medicine, space science, educa- 
tional research, and library science. (3) 

The projects described below are examples of the informa- 
tion problems that are currently being solved. Many of the in- 
sights and techniques developed in dealing with published material 
in a given field are applicable to another field with appropriate 
modifications. 

The "Knowledge Availability Systems Center" at the 
University of Pittsburgh has assumed the task of determining how 
aerospace data can best be stored, processed, made readily avail- 
able, and quickly utilized. Its objective is to analyze more than 
3,000 documents on aerospace research that are published each 
month. Faculty members conversant in engineering, natural sciences, 
and library science help to interpret the needs of local companies 
and to pinpoint applicable research findings. Swift computerized 
searches for relevant information are made through the facilities 
of the Computation and Data Processing Center at the University. 

The core of its activities include, on a monthly basis: 

1. Updating more than 3,000 documents on current 
aerospace developments. 

2. Abstracting all documents. 

3. Compiling a cumulative index to the documents, 
stored on tape for computer retrieval. (5) 

In the field of education, the pioneering efforts of the 
Center for Documentation and Conxnunication Research at Western 
Reserve University are especially relevant. In 1961, the U. S. 
Office of Education contracted with the Center to develop a pilot 
information service for educational media research applied generally 
to other areas in educational research. The need for a "system" 
was convincingly demonstrated by the Science Information Exchange. 

By early 1963, it had produced abstracts of 686 current research 
projects in education. A conservative estimate of $20,000 per pro- 
ject, totaling over $1,370,000 for all projects, indicates the 
measure of the confidence evidenced by the supporters for the basic 
usefulness of the method. (2) 



Analysis and retrieval of biomedical literature is the con- 
cern of an arrtoitious system installed at the National Library of 
Medicine in Bethesda, Maryland. The system is considered the 
world's largest collection of journals, bocks, and monographs in 
medicine available to physicians , scientists , and medical educators . 
There are three chief products of MEDLARS (Medical Literature 
Analysis and Retrieval System) : 

1. A monthly listing of current biomedical literature, 
called Index Medicus , consisting of 14,000 refer- 
ences. 

2. Bibliographies in such specialized subjects as 
cancer, heart disease, dentistry, and medical 
education. 

3. Demand bibliographies which result from requests 
for machine searches of the computer ' s data files 
for answers to complex reference questions. 

Each day about 700 new articles are processed and entered 
in the files. The current data base consists of about 450,000 
articles. Information is derived from 2,400 journals received at 
the library from all over the world. (4) 

Another application of information retrieval is the project 
at the University of Pittsburgh where the Pennsylvania statutes and 
the attorney general's opinions relating to education have been 
placed on magnetic tape. In addition, all of Pennsylvania statutes, 
some 75 volumes, have been put onto tape. This information was used 
to generate a concordance which alphabetized all the words in the 
library and shewed their frequency of occurrence. The system at- 
taches to each word a series of numbers which identifies the exact 
location in the library where each occurrence of the word appears. 

By knowledge of the frequency of occurrenoe of various words and the 
words representing key concepts in the legal questions, a strategic 
approach can be developed which will yield a maximum number of 
valuable documents and keep at a minimum the number of irrelevant 
or semi-relevant documents retrieved. (1) 



PURPOSES 

PRIMES proposed to accomplish these objectives: 

1. Develop a system for analyzing, storing, and retrieving 
information related to elementary school mathematics 
curriculum materials. 
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2. Develop procedures for utilizing the system for cur- 
riculum decision-making by local school districts in 

a. Constructing curriculum guides - An elementary 
school mathematics curriculum coirmittee can use 
the file components in determining a scope and 
sequence of content with suggestions for teach- 
ing the content that reflects the philosophy of 
the local school district. 

b. Selection of a basal textbook series - The file 
of definitive information on existing published 
programs can be used by local school districts 
as an aid in selecting those instructional 
materials that are most appropriate for imple- 
menting their curriculum guide. 

c. Selection of supplementary material - Use of the 
curriculum file can assist a school district in 
identifying the content and behaviors for its 
present text series and locating instructional 
material containing concepts that are not in- 
cluded. Additional materials can be provided 
for review work , remedial instruction, a dif- 
ferent mode of presentation, and more challeng- 
ing work for the more able students. 

3. Evaluate the effectiveness of the system in accomplish- 
ing its intended purposes by determining 

a. The utility of a curriculum information system - 
Working with local school districts in "real life" 
situations was to provide PRIMES staff with a set 
of specific questions that were actually addressed 
to the system. Careful documentation and assess- 
ment of the way in which the system was used to 
answer inquiries would guide the further develop- 
ment of the system. 

b. Degree of user satisfaction - Several school 
districts were to be studied to determine present 
procedures in curriculum decision-making and the 
ways in which the system affected, these procedures. 
The data was to be used to evaluate this approach 
to curriculum decision-making and to provide 
guidance for changes required in a full-scale pro- 
gram. 
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c. Economic and tedinical feasibility - implying 
information retrieval technology to education 
requires careful stud/ of costs to determine 
services that are technically and economically 
feasible. 
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METHOD 



PRIMES has designed and implemented detailed proce- 
dures for system development and system utilization. 



SYSTEM DEVELOPMENT 

f 

Unit Document 

•* The system document consists of three basic elements - 

the analysis sheets, the teacher manual page, and the pupil 
text page (Figure 1) . Selected characteristics of each 
textbook of twenty published elementary mathematics basal 
series have been analyzed, lesson-by- lesson. The analytical 
categories are: 

Content - Each lesson is classified according to the 
mathematics concepts presented. A master list of more 
than three hundred content items, classified under 
seven general mathematics topics, is used in assigning 
a content code number. 

Expected pupil behavior - Each lesson is assigned 
specific objectives. Approximately fifteen hundred 
objectives, stated in behavioral terms, are contained 
in the master list used in lesson classification* 

Type of problem - The problems and examples included 
in each lesson are classified according to four main 
types: direct application of lesson, review, ex- 

tended application, and exploratory activities . The 
format, computational or problem solving, is also 
indicated. 

Vocabulary and synfaolism - Whenever a technical term 
or synfool is first introduced, it is entered on the 
analysis page and noted on the teacher’s page for that 
lesson. 

5. Grade level - All lessons are coded according to the 
school year and month for Which they are intended. 

6. Pre-text activity - Suggested preparatory activities 
described in the teacher manual are identified. 

7. Post-text activity - Suggested activities for lesson 
follow-up are identified on the teacher’s page. 



1 . 



2 . 



3. 
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Analysis 

The analysis procedures for producing the system document 
are based on standards developed by specialists in mathematics 
education, educational psychology, curriculum, and information 
sciences and technology. The actual analysis was done by edu- 
cators experienced in doing the type of analysis required by 
the system. The analysis responsibility was divided among four 
groups of people: the content analysts, the behavior analysts, 

the problem-type analysts, and the features editor. Work began 
at Harrisburg where the project staff identified the programs 
to be analyzed, assigned the analytical work, and sent copies 
of the teacher manuals for each program to the respective analyst. 

The content analysts were professors of mathematics at 
colleges or universities in the Commonwealth. These analysts 
were trained and their work was supervised by the senior editor 
of the project. The analytical work consisted of identifying 
the mathematics content of each lesson and recording the appro- 
priate codes on the book itself. The codes were selected from 
the Content Authority List which was the analytical tool for 
this classification. The list of about three hundred mathe- 
matics concepts and skills was developed by a committee of 
nationally recognized mathematics educators. 

Similarly, the behavior analysts studied every lesson of 
each basal textbook and assigned one or more behaviors that -the 
pupil is expected to demonstrate as a result of the indicated 
instructional activities. The master list of behaviors. 

Behavior Authority List, was based on the mathematics curriculum 
research work done at the Learning Research and Development 
Center, University of Pittsburgh. The list, of approximately 
fifteen hundred pupil objectives, was applied to each lesson by 
seven classroom teachers trained and supervised by the authors 
of the list. The analysts recorded the appropriate behavior 
codes on the bock itself and sent the books to the supervisor 
for review. 

The problem-type analysts assigned problem- type codes 
directly on a copy of the teacher's manual. He also indicated 
pre- and post-text instructional activities and identified tech- 
nical vocabulary and symbolism by underlining the words and 
symbols on the teacher manuals. A vocabulary-symbolism 
alphabetized summary index was kept for each basal textbook 
series and was included at the end of the document file for 
that textbook. This analytical work was done by two classroom 
teachers who were trained and supervised by the project staff. 
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Hie features editor reviewed each textbook to identify 
those salient features which could be documented and prepared 
a written draft description of the features of each program. 
Hie features of the pupil textbook and teacher manual con- 
tent and organization and the scope, sequence, and develop- 
ment of mathematics content were described following an 
outline developed by the editor (Figure 2) . This draft was 
typed and sent to the publisher for review, after which it 
was re-edited and a final draft prepared. Hie features 
section was also included at the end of the document file for 
each textbook. 



Analytical Tools 

Hie Content Authority List (Figure 3) is a list of 
mathematics concepts and skills that may be taught from 
kindergarten through grade six. Hie list is arranged in a 
hierarchical order with each topic identified by a unique 
nurrber. 

Hie second authority tool is the Behavior Authority List 
(Figure 4) that is logically related to the mathematical con- 
tent list. There are eleven mathematics topics in five major 
levels of skill-activity difficulty, A-E. The specific be- 
havioral statements are classified under one of these eleven 
topics . 

, The problem types were defined by the same mathematics 
educators who developed the Content Authority List. Hie four 
basic categories are: (1) direct application of the lesson, 

(2) review, (3) extension of the lesson application or general- 
ization, and (4) exploratory, preceding formal teaching. The 
first category was coded A; the second, B; the third, C, and 
the last, D. Each problem was further described for its format 
with computation coded 1 and problem-solving 2 (Figure 5) . 

A master alphabetized list of technical vocabulary- 
symbolism was provided for the analysts . This list was greatly 
expanded because nonstandard vocabulary and symbolism may be 
indicated as integral to the developmental philosophy of a 
particular basal program. This was determined by the analysts 
after thoroughly studying the overview of the program and the 
manual discussion for each lesson. Numerical codes were not 
used for the vocabulary-symbolism. An index was compiled for 
each textbook (Figure 6) . 



FIGURE 2 



OUTLINE FOR FEATURES 

Features Checklist: Development of Content 

Points to be Noted 



1. Counting 

2. Cardinal Numbers - meaning and writing of; sequence of development 

3. Ordinal Nurrbers - meaning and writing of; sequence of development 

4. Place Value 

5. Addition - meaning - where and how developed 

Subtraction 

Multiplication - computation - where and how developed 
Division 

6. Measurement - length, time, liquids, money, etc. 

7. Fractions - meaning - where and hew developed 

8. Word Problems - where and hew developed - use of number sentences 

9. Emphasis on Concepts and Skills 

10. Provision for Individual Differences, including Enrichment 

11. Reviews 

12 . Tests 

13. Some "Modem" Topics and Procedures 

a. Use of discovery and inquiry 

b. Equalities and inequalities 

c. Development of generalizations 

d. Geometry 

e. Number line 

f . Numerals as names for a number 

g. Odd and even numbers 

h. Properties of operations 

i. Properties of one and zero 

j. Regions - interior and exterior 

k. Sets 

l. Use of precise vocabulary 



FIGURE 3 

SAMPLE PAGE - CONTENT AUTHORITY LIST 



Mathematical Content Authority list: K-6 October 1967 

0002. . . Topic I: Nunfoer Systems 



0004 A. Whole numbers 

0006 1. Basic concepts 

0010 a. Definition: set of Whole nurtbers 

0020 b. Developing cardinal nurrber zero 

0030 c. Developing cardinal nurtbers one through 

ten 

0035 d. Developing cardinal nurtbers beyond ten 

0040 e. Developing ordinal nurrber sense 

0050 f. Associating idea of nurrber with nurrber 

line (one-to-one correspondence) 

0060 . . . . g. Counting to find cardinal nurrber of set 

(one-to-one correspondence) 

0070 h. Ordinal counting 

0075 i. Sequence of nurtbers increasing by one 

0080 j. Skip counting 

0090 k. Other counting: backward, rote, etc. 

0100 1. Ordering nurtbers and objects as greater 

than? less than; equal to 

0102 2. Operations 

0104 a. Addition 

0106 1) Properties 

0110 a) Definition: addition, a binary 

operation 

0120 b) Addition developed from union of 

disjoint sets or joining action 
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FIGURE 4 

SAMPLE PAGE - BEHAVIOR AUTHORITY LIST 
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FIGURE 5 

SAMPLE PAGE - PROBLEM TYPE 
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FIGURE 6 

SAMPLE PACE - VXABULARY INDEX 

04575 

American Book Company 
Bock One Meeting Mathematics 1965 
Deans . Kane . Oesterle 

-A- 



1. add 


TG5 


27. fewer than 


TG4 


2. addends 


TG57 


28. fewest 


TG6 


3. addition 


P108 


29. fifth 


TG3 


4. after 


TG4 


30. fifty 


P132 


5 . altogether 


TG2 


31. first 


TG3; P2 


-B- 




32. five 


P7 


6 . before 


TG4 


33. foot 


TG52 


7. between 


TG4 


34. forty 


TG66 


8. bottom 


TG2 


35 . four 


P7 


-C- 




36 . fourteen 


P174 


9. cent 


TG59; P112 


37. fourth 


TG3 


10 . cent sign 


TG59 


38. fourths 


TG71 


11. clock 


TG72; P141 


-G~ 




12 . clockwise 


TG72; P141 






13. count 


TG63; P119 


-H- 




14. cup 


TG52 


39 . half dozen 


TG87 


-D- 




40 . half hour 


TG85 


15. day 


TG36 


41. half past 


P178 


16. different (sets) 


TG3 


42 . highest 


TG48; P85 


17. dime 


TG59; P112 


43. hour 


TG72 


18. doubles 


TG43; P72 


44. hew many 


TG2; PI 



15 



Grade level was determined by the number or letter as- 
signed by the publisher. The month was determined by dividing 
the number of pupil pages by the nurrber of school months (9) . 
Two-digit codes were used, the first indicating grade and the 
second indicating month. For example: 21 is Grade 2, 

September; 65 is Grade 6, January; 07 is Kindergarten, March. 

Pre-text activities were noted on the teacher manuals 
with a single check and a single squiggle. Similarly, the 
post-text activities were marked with a double check and 
squiggle (Figure 7) . 



Production 

Materials were prepared for microfilming at Harrisburg. 

The analytical data, content, behavior, problem type, and 
vocabulary-symbolism codes were conformed on a summary form. 
Lesson Summary Sheet (Figure 8) . The content and behavior 
codes were verified for validity and reasonableness. At this 
time the document number, accession nurrber, and the grade level 
were assigned. 

From the lesson Summary Sheet the numerical codes were 
transcribed to lesson Keypunch Forms (Figure 9) . The forms 
were sent to Bureau of Educational Data Processing where they 
were keypunched, interpreted and listed for computer input. 

The keypunched cards were then used to generate the print 
analysis pages and the master indexes. 

The teacher manuals and pupil textbooks were cut, collated, 
and stamped with an accession number and problem type. The next 
activity was the transcription of pre- and post-text activities 
notations from the analyst copy to a clean camera-ready copy of 
both the pupil and teacher books. The analysis pages were 
proofread and sent back for correction if needed. Symbols that 
were not printed by the computer were indicated by an asterisk 
and handwritten. 

The computer-printed analysis page, the noted manual pages, 
and the pupil textbook pages were then collated and prepared for 
microfilming. lessons that required multiple frames, i.e., 
trailer cards, were identified during this copy preparation work. 

The features and the vocabulary summary were typed. The 
vocabulary summary includes the page number at which the vocabu- 
lary is first introduced. The typed copy of the features was 
sent to the publisher for editorial review. 
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FIGURE 7 

SAMPLE PAGE - PKE-/POST-TEXT ACTIVITIES 



Addition and subtraction: sums of 7 



20 41 4 

To relate subtraction facts to the 
f corresponding addition facts. 

"p&e-bodk teaching 

1. Make and use flap cards as de- 
scribed on page 70 with the following 
dot patterns on them: 6 and 1; 5 and 
2; 4 and 3; 7 and 0. 

2. Ask the children to tell you 
(without the aid of pictures) what 
other number sentences belong in the 
family of each of these sentences. Re- 
cord each family on the chalkboard. ( 

6+1 = 7 4 + 3 = 7 

5 + 2 = 7 7 + 0 = 7 




PROCEDURE 

Discussion 

Do the page orally, first calling at- 
tention to the number sentence fami- 
lies in Ex. 1 to 3. Have children state 
the whole number sentence, not just 
the answer, in all exercises. 

You may wish to have several chil- 
dren demonstrate some of the sen- 
tences on the number line. 

Written Work 

Pupils may now proceed independ- 
ently to write their responses to the 
exercises on this page. 



£Ui 



JF.MEN fARY 




2opy the following on the chalk- 
board and have the children tell what 
signs belong on the colored blocks. 
Note the last two sentences, in one of 
which a + sign may be used, and in 
the other a — sign. 



7 




4 


= 


3 


2 


% 


4 


= 


6 


7 




7 


= 


0 


7 




0 


= 


7 


7 




0 


= 


7 



2. Children enjoy playing th s game 
described below. 

Game 

Stranger in the Family 

Place the following rows of addi- 
tion and subtraction facts on the 
board. Point to the first row and say: 
In this row of number facts there is 
one fact that does not belong. There is 
a stranger in the family. What is it? 
After the answer 7 — 6=1 has been 
given, have the pupils find the 
stranger , in each of the other rows. 

(a) 



(b) 



(c) 



(<!) 



3 


4 


7 


7' 


7 


+ 4 


+ 3 


-4 


- 6 


-3 


7 


7 


3 


1 


4 


7 


1 


7 


6 


6 


-6 


4 ~ 6 


- 1 


- 1 


+ 1 


i 


7 


6 


5 


7 


4 


2 


6 


2 


6 


+ 2 


+ 2 


-4 


+ 4 


-2 


6 


4 


2 


6 


4 


5 


5 


2 


7 


7 


+ 1 


4 “ 2 


4 - 5 


-5 


-2 


6 


7 


7 


2 


5 



For Faster Learners 

Complete these number sentences: 

(6-3) +4 = □ 

(7 — 2) + 1 
5 + (7 
□ + (6 
(7-3) 
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The camera-ready copy was sent to a commercial company for 
microfilming. The original film is on 35 -mm rolls which are 
stored. Delicate sets of the film are produced on aperture 
cards and rolls which are stored at the Department (Figures 10, 
11 ). 



A system for utilizing the computer to prepare the analy- 
sis pages, authority lists, and indexes resulted in considerable 
reduction of clerical effort and error (Figures 12 , 13) . The 
primary inputs to the computer system were the numerical codes 
recorded on a prescribed form. Lesson Keypunch Form. The other 
input was changes made to the behavior and content lists. The 
output from the computer system included the camera-ready copy 
of the lesson analysis sheets, a complete set of punched cards 
representing the index card file which is searchable on elec- 
tronic sorting equipment, printed indexes for manual searching 
arranged by content and behavior, and updated behavior and 
content lists. 

The authority lists have been coverted to machine-readable 
form and were maintained on a magnetic tape file which can be 
printed on demand. Changes can be readily made on these lists. 

Cumulative book-form indexes were generated by computer to 
aid searching. The printed indexes represent a combination of 
machine-readable authority files and index files. The index 
first lists the numerical code and the category heading in 
English language which is then followed by a list of the code 
numbers and accession numbers which fall under that category. 

Use of the computer has made periodic updating of the , indexes 
possible (Figures 14, 15). 
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FIGURE 10 

ANALYSIS AND COPY PREPARATION - WORK FLOW 
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FIGURE 11 

MICROFILMING AND FILE MAINTENANCE - WORK FLCW 
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FIGURE 12 

COMPUTER-AIDED WORK FLOW 
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FIGURE 14 

SAMPLE PAGE - CXKCENT INDEX 



CONI ACCESS 


9HVR 


LEV 


PE 


07000 1* ODD-EVEN 


007400 


06120 


27 


AX 


007420 


05080 


27 


A1 




05220 


27 


B1 


007430 


05160 


27 


A1 


C0744C 


06120 


27 


A1 


0461Q0 


01980 


21 


A 1 


164420 


06120 


18 


A 1 


166650 


06120 


14 


A1 


203770 


06040 


15 


A1 


2055080 


06120 


31 


A 1 


06240 


31 


A 1 


363830 


06120 


14 


Al 


366270 


06120 


12 


A1 


366990 


06120 


15 


Al 


446350 


06120 


23 


Al 


484110 


02300 


17 


A 1 


484120 


06120 


17 


Al 


486430 


06120 


22 


Al 


486610 


06540 


24 


Al 


486620 


06120 


24 


Al 


486630 


06120 


24 


Al 


604490 


06120 


18 


A 1 


647750 


06120 


28 


Al 


686140 


01960 


22 


Al 


C7090 It ARITHMETIC 
284250 


PROGRESSIONS 

06020 


18 


Al 


284280 


06240 


19 


Al 


284290 


06240 


19 


Al 


284310 


06240 


19 


Al 


284320 


06240 


19 


Al 


284330 


06020 


19 


Al 


404590 


06240 


19 


Al 


407360 


06240 


28 


Al 


407370 


06240 


28 


Al 


07110 3, TRIANGULAR 

123410 


NUMBERS 

06160 


14 


Al 


07140 6f GRID GA M ES 


046470 


02040 


23 


Al 


04743C 


04800 


28 


Bl 


206980 


04800 


26 


Al 


07160 C# SPECIAL PATTERNS (INCLUDING 


SHORT CUTS) 


Al 


" ' 123080 


00390 


11 


123650 


06360 


16 


Al 


123730 


06360 


17 


Al 


123810 


08240 


17 


Al 


123880 


06360 


18 


Al 


123890 


06360 


18 


Al 


123910 


06240 


18 


Al 


CONT ACCESS 


BHVR 


LEV 


PE 



FIGURE 15 

SAMPLE PAGE - BEHAVIOR INDEX 



BHVR 

01800 



01820 



0X840 



LEV 


ACCESS 




PE 


CONT 


6. PROVIDES OR COMPLETES AN 


ADDITION NUMBER 


S£N- 


tence using the * 


AND e SIGNS FOR 


PICTURES COMPOS 


ED OP GROUPS OP OBJECTS OR 


SETS 


WRITTEN IN 


N NO* 


TATION, responds 


BY WRITING 


THE 


NUMERALS* 


SUM 


TO 20* 










23 


406450 




A1 


00120 




566670 




AX 


00190 




566680 




AX 


00190 




566700 




AX 


00190 


24 


206650 




A1 


00150 

00310 




446370 




Al 


00150 


25 


446500 




A1 


00190 




446600 




Al 


00150 




606800 




A2 


00150 

00270 


26 


446660 




Al 


00150 


28 


446880 




A2 


00310 


7# CONSTRUCTS OR 


COMPLETES 


THE APPROPRIATE 


SETS* 


, POR SUMS TO TWELVE* WHEN GIVEN 


A NUMBER SENTENC 


E CONTAINING A + 


SIGN AND " 


SIGN* 




09 


441310 




Al 


00030 

03050 


11 


603210 




Al 


00120 




603220 




A2 


00120 




603230 




Al 


00120 


12 


603310 




Al 


00120 




603320 




Al 


00120 

00190 


13 


043450 




Al 


00120 

00250 


14 


403690 




Al 


00190 

00310 




403730 




Al 


00190 

00310 


15 


003880 




Al 


00190 


403810 




Al 


00190 

00310 




16 


004040 




Al 


00120 




004130 




AX 


00190 




283900 




Al 


00310 


17 


004220 




Al 


00120 




404210 




Al 


00190 


18 


004400 




Al 


00190 


21 


606170 




Al 


00120 



SYM.8* SUPPLIES THE CORRECT SIGN# « OR *Q FOR A- 

ddition statements using numerals and ♦ sign* on 

E-STEP EQUATIONS AND SUMS TO 12* 

13 283460 B1 09000 



LEV 



BHVR 



ACCESS 



PE 



CONT 



SYSTEM UTILIZATION 



Providing local school districts with the services of 
PRIMES presented a major problem - the products were too complex 
for local school district personnel to use effectively without prior 
training in curriculum development work and system utilization. 
PRIMES developed a plan to resolve this problem. Two basic issues 
had to be considered: (1) Should servicing school district be 

centralized or decentralized? (2) What procedures for utilization 
of the system could most effectively satisfy the needs of the 
districts? 



A totally centralized operation would provide for all of 
the functions at one location. Arguing against this type of 
organization are the problems imposed by peak loads, professional 
staffing requirements, and the amount of time required for travel. 

An integral aspect of the system is a dialog between the user and 
the reference specialist to ensure thorough communication of the 
intent of the search question. Ideally, this takes place best when 
the user and the reference specialist are face-to-face. If this is 
not possible, telephone communication is a second choice. A second 
system requirement which favors decentralized operation is the need 
for evaluating each document retrieved by the index search function 
for relevancy to the question. While this is performed at a first 
level by the reference specialist, it must ultimately be performed 
by the user. Hence, if the information center is sufficiently close 
to the user, it would be possible for the user to utilize the system 
in person. 



For these reasons it was decided that service to users be 
provided by regional centers located in areas convenient to user 
groups. The number and location of these centers will be determined 
by considerations of available organizational facilities, financial 
support, and trained staff. 

While it is more efficient and effective to provide the 
reference services on a decentralized basis, conversely it is more 
efficient to prepare the input to the system on a centralized basis. 
The cost of analysis and input to an information retrieval file is 
substantial . Therefore, it should only be performed once which does 
not imply that the input must all be physically prepared in the 
central location. The intellectual functions should be performed by 
those best qualified wherever they are located. However, the co- 
ordination as well as the physical input processes should be done 
centrally, as well as the duplication and updating of all files for 
use in the regional centers. 
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A Plan for Regional Centers 

A design for establishing regional centers was completed. 

It includes: 

1. A master plan for the design, development, and im- 
plementation of a model regional center. 

2. The functions to be performed, the files to be main- 
tained, the inputs required, the products and services 
to be produced, the output formats, the interface with 
the central facility, and specifications for any 
special programs required. 

3. The types and number of personnel required to staff 
the regional center. 

4. The manner in which questions are received, the records 
that are to be maintained, the services that are to be 
provided, the procedures for file maintenance, the de- 
velopment or identification of special reference tools, 
and the mode of and procedures for communicating with 
the user. 

5. Procedures for training the staff complement and neces- 
sary training tools. 

6. Orientation programs for users to acquaint them with 
the availability of service and also to acquaint the 
users with how they can best utilize the system. 

7. On-site checkout of system visits to insure that the 
system is operating in accordance with the system 
design specifications. Where problems are identified, 
recorrmended solutions are developed. 

8. Procedures for measuring and evaluating the performance 
of the system at regional centers. 

9. A procedures manual to provide a precise, systematic 
plan for users to construct curriculum guides and 
select appropriate instructional materials using the 
system. The manual describes the curriculum cannittee 
composition, its fifteen discrete tasks to accomplish 
the two objectives, the time schedules for implementing 
the tasks, and the final products that should result 
from this curriculum effort. 



Processing Inquiries 

The first activity in establishing the service capabili- 
ties of a regional center is an orientation by the curriculum adviser 
for the users explaining hew the system can assist them in curriculum 
development. This curriculum consultation would take place over a 
period of several. weeks or months. The users of the system would be 
invited to submit questions which could be answered by the information 
system. The questions would be addressed to the system by telephone , 
a letter, or a personal visit. Upon receipt of the question, a 
staff member would record it on a pre-printed form, log it in, and 
stamp the date of receipt as a basis for getting statistics on the 
total time it takes to process the question. The question would then 
be assigned to a reference specialist. 

The reference specialist would formulate the search query 
in the restricted language of the system. This means that the various 
concepts included in the question must be coded according to the be- 
havior and content lists, problem types, grade levels, etc. He may 
have to contact the laser to clarify the exact intent of the question 
to insure that the system responds with an answer to the user's question 
and not to a hypothetical question which the reference specialist may 
otherwise assume to be the problem. 

The reference specialist will scan an index to completed 
searches to determine whether or not there has already been a similar 
search. If one is found, the reference specialist will update or 
modify the search as required. The search can be performed irdng the 
printed indexes. The searcher will scan for either a behavior code 
or a content code and manually examine the entries under that code 
for the presence or absence of other codes in the question according 
to the search logic. The output of the search process will be a list 
of accession numbers of the lessons which satisfy the criteria of the 
search question. The aperture cards from the document file will be 
pulled by accession nuirber. 

The aperture card is then inserted into a microfilm reader 
and reviewed to access relevancy of the particular item or document. 

If the search is satisfactory, the answer to the question is prepared 
and sent to the user. Answers may take one of several forms - a list 
of page numbers referring to the original textbook, a narrative, micro- 
film duplicates, or a copy of the unit document printed out on the 
reader-printer . 

After completion of the search, the question is indexed so 
that the results may be used to satisfy future search questions 
without the intervening steps of scanning the index and analyzing the 
documents themselves. The completed search index file is then updated 
and the completed search, including a copy of the answer sent to the 
user, is filed in the completed search document file. 
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Special studies of curriculum matters of general interest 
are handled in much the same way and are indexed as conpleted searches . 
The primary difference, however, is that the results to these ques- 
tions are mass-produced by some form of duplication. Rather than 
filing a single copy of the answer, as in the case of an individual 
search question, a supply of copies is stored. Similarly, the special 
studies may be prepared in advance for questions which may be of 
interest to a large nunfoer of users (Figure 16) . 
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FIGURE 16 
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RESULTS 



The results discussed in this section concern the 
activities of PRIMES from January 1966 through June 
30, 1967. Developmental work and servicing local 
school districts are continuing activities. These 
experiences lead inevitably to products modifica- 
tions and procedural changes which will be described 
in future reports. 

It should also be noted that this section will 
describe the products, services, and application of 
PRIMES; it contains little empirical data to sup- 
port the contention that PRIMES is indeed a more 
effective system for curriculum development work 
than traditional methods. 



PRODUCTS 



PRIMES developed several tools which can assist local 
school districts in curriculum development activities and selection 
of appropriate instructional materials. These tools are; 

Content Authority List . This list consists of approxi- 
mately 300 mathematics concepts and skills that may be 
taught from kindergarten through grade six. It was de- 
veloped by a committee of nationally recognized mathe- 
matics educators. The items are arranged in hierarchical 
order with a unique number assigned to each of the items. 
Generic searching is possible by scanning the list which 
is organized in an English outline format. Additional 
items to the list are assigned numerical codes using the 
fourth digit, e.g., an insert between 0030 and 0040 is 
accomplished by using the number 0035 (Figure 3 - METHOD) . 

Behavior Authority List . This list consists of approxi- 
mately 150Q objectives for elementary school mathematics 
stated in behavioral terms. The basis for the list was 
the curriculum that was developed for the Individually 
Prescribed Instruction (I.P.I.) Project at the learning 
Research and Development Center, University of Pittsburgh . 
There are eleven mathematics topics in five major levels 
of skill-activity difficulty designated by the letters A 
through E. The entries in this list are more numerous 
than those in the Content Authority List. The items are 
not organized in as highly a structured order as those in 
the Content List. A six-digit code is assigned to each 



o 

ERIC 



32 



behavior. Provision has also been made for adding new 
items to the list. 

Indexes . There are two types of master indexes in the 
system. In the Master Content Index the content items 
are used as a reference. These are listed by nunber and 
in English language. All the accession numbers to which 
each item in the Content Authority List has been assigned 
are listed in sequential order in the first column. 

Numerical codes in parallel columns indicate the behav- 
ioral objectives, grade level, and problem types for each 
accession number. The Master Behavior Index is similarly 
arranged with the analytical information indicated in this 
order - grade level, accession number, problem type, and 
content. These two indexes are the tools used in identify- 
ing alternative source materials for teaching specific 
concepts or in comparing programs (Figures 14, 15 - METHOD) . 

Vocabulary and Symbolism Lists . The system has compiled 
a draft of a master alphabetized list of technical vocab- 
ulary-symbolism which is continually being updated as new 
items are noted in the course of analysis. In addition, a 
vochbulary-symbolism summary index has been compiled for 
each basal textbook series and is part of the document file 
for that textbook (Figure 6 - METHOD) . 

Features . In addition to the lesson-by-lesson analysis the 
system includes a description of the significant features 
of each textbook. The features are presented in an objective 
style and are documented with a page reference. An outline 
developed by the editor has been followed in describing 
such characteristics as the organization, scope, sequence, 
and the development of the mathematics content. The final 
version incorporates suggestions that were made by the 
publishers and accepted by the editor. The features are 
filed with the document file (Figure 17) . / 

Microfilm File. The document file consists of aperture 
microfilm cards and roll microfilm. When complete, the 
microfilm aperture card file will exoeed 25,000 cards. 

Each card has an accession number identification and the 
grade level at which the lesson is taught . The unit 
document has been described in the METHODS section (Figure 1) . 

Original published textbooks . The system has all the 
original teacher manuals and pupil books that have been 
analyzed and filed in microform. 
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General Organization 



FIGURE 17 

SAMPLE EXCERPTS - FEATURES 



The Pupil's Text 



The Pupil’s Text contains 96 large (10 x 14 inches) work 
pages. The first 53 pages are printed in bright orange, blue, gold, 
and black. Color is an integral part of the development of the pre- 
nurrber ideas. On these pages pupils are required to draw lines to 
shew that colored triangular, square, and circular regions belong in 
certain colored shapes. In the last 43 pages additional color is used 
to order sets and introduce cardinal nurrbers one through nine. On 
these pages pupils are required to: 

1. draw lines to show the matching of sets (pp. 55,59,69) 
or the path of a frog (pp. 73,74) . 

2. draw forms to make equivalent sets (p. 63) . 

3. mark an X on the proper set (pp. 65,87) . 

4. draw a ring around pictured sets (p. 67) or 
numerals (p. 75). 

5. recognize the size of a group and the numeral for its 
number (p. 83). 



The content is developed through games for which children are en- 
couraged to make the rules (T-v) and in which they become accustomed to 
handling progressively more difficult situations. 

Be foie the pupil uses his text he must knew the names of the colors 
blue, gold, orange, and black and recognize them. He must also knew the 
forms: circle, square, and triangle. The authors believe that a pupil 

can then proceed at his am rate throughout the text (T-l, 44) . 



Points to Note 

4. The suggested matching games, construction games, finding 
games (T-xv,xvi) , placing games, and classification games 
(T-97) . 

5. The emphasis on independent work by the pupils (T-31,40) , and 
on pupil explanation of why each mark was drawn (T-2,4,40) . 

The teacher is directed to check the pupil’s work carefully 
(T-27,39 ,73) . 

6. The intuitive development of interior regions (T-48 continued) . 

7. The intuitive development of set concepts (T-6,48 continued). 

8. Provision for additional activities of a more challenging 
nature though related to the page activity (T-28,63) . 
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SERVICES AT REGIONAL CENTERS 



PRIMES has been instrumental in establishing two regional 
centers to offer its services to local school districts. The first 
center established was the Westmoreland Regional Instructional 
Materials Center at Greensburg, Pennsylvania. The center is funded 
under an ESEA Title III grant and began operations in January 1967. 
The second center was established in July 1967 at State College, 
Pennsylvania. Together these regions serve a seven-county area with 
a total of fifty-one school districts. 

Types of services. Three levels of consultative curricu- 
lum services are offered to the school districts in the regional 
center's service area. The types of service are tailor-made to the 
needs of the local school districts. These services are: 

In-depth - this service refers to a comprehensive 
study of the school's current elementary mathematics 
curriculum, constructing a new curriculum guide, and 
selecting appropriate instructional materials to 
implement this guide. The center carmits its staff 
and resources to work with these school districts 
over an extended period of time. It is expected that 
the cooperating school districts will be given the 
necessary time and facilities to work on the project. 
The center staff works with the school district on a 
regularly scheduled basis. 

Modified - this service involves the analysis of a 
school district's present mathematics program. The 
PRIMES file can be used to analyze the content and 
objectives that are currently being taught in the 
school district at each grade level. The school 
district may utilize this analysis to develop in- 
service workshops, to select tests that are related . 
to the current textbook program, and to purchase 
instructional materials from other series that will 
supplement the current program. 

General - the center offers a general service to all 
school districts in its service area to strengthen 
the elementary school mathematics curriculum. This 
would include services such as answering queries, 
advising on developing an in-service program, and 
locating personnel for conducting workshops. 
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Personnel . Each center has a project director, a mathe- 
matics curriculum adviser, and clerical staff. Providing the 
curriculum services is the responsibility of the mathematics adviser. 

Facilities. The regional center provides a meeting room 
for individual study and for group meetings. It also has sets of 
the book-form indexes, the master authority lists, the aperture card 
file, sets of features, the original published basal series, sample 
curriculum guides, samples of criteria used in textbook selection, 
a microfilm reader-printer, and duplicating equipment. 

Cooperating schools. Five school districts are receiving 
in-depth service and are utilizing PRIMES materials. They are Penn 
Trafford, Franklin Area, Greensburg-Salem, Kiski Area, and Southnore- 
land. These are all being serviced by the regional office at 
Greensburg, Pennsylvania (Figure 18) . 



APPLICATION OF PRIMES 

Four programs, utilizing PRIMES, have been conducted for 
school personnel: 

Workshop for Elementary School Personnel. A two-week 
workshop, June 19-30, 1967, was conducted at the Greensburg Center, 
University of Pittsburgh, to train teachers and curriculum coordi- 
nators to apply PRIMES in curriculum decision-making. Graduate 
credit was given for participating in this program. Participants 
were taught the mathematics concepts in the Content Authority List. 
Laboratory experience was also provided in searching the file, 
selecting and studying the aperture cards, and analyzing the 
original published materials. Curriculum development activities 
were simulated and selected lessons were sequenced for individual- 
izing instruction. 

Pre-service Training. Two of the PRIMES analysts have 
informally utilized the file and the indexes in their undergraduate 
course in the teaching of elementary school mathematics at West 
Chester State College and Indiana University of Pennsylvania. 

Conference for Secondary Teachers on Elementary Mathematics , 

Indiana University of Pennsylvania. This conference was 
organized to train secondary mathematics teachers to service as coordina- 
tors of mathematics teaching in the elementary schools. PRIMES was 
presented to them and they were invited to utilize its services in 
their future activities. 
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FIGURE 18 

PRIMES REGIONAL CENTERS 
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Institute for Instructors of In-service Education in 
Mathematics . One-week institutes were conducted at three 
of the state colleges for secondary or elementary teachers who have 
had recent college courses in mathematics. These participants were 
instructed in procedures and materials for conducting in-service 
workshops for the elementary teachers of their local school dis- 
tricts. An orientation session to PRIMES was conducted at each 
institute and the participants were invited to utilize the products 
and services of the system in their in-service teaching responsi- 
bilities . 
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DISCUSSION 



This study is a description of the development of 
products and their application in servicing local 
school districts. Limited experience with PRIMES 
in dealing with curriculum problems precludes a 
discussion of the system's effectiveness. 



LIMITATIONS 

Limitations that are discussed are those which PRIMES staff 
anticipates in utilizing the file. The staff has consulted with ex- 
perienced people in the field of teacher training and curriculum 
development. The general evaluation is that the materials are far 
more extensive and promising than anything that has yet been developed. 
Its effectiveness in affecting curriculum development work will be 
determined by the ability of school personnel to use the system. 

The specific areas in which there is need for additional ex- 
perience are: 

Training of personnel . To use the file effectively it is 
necessary that elementary school personnel understand both 
the master authority lists. This would require an in- 
service workshop consisting of about 30 hours of instruction. 
Extensive use of the file would present a considerable lo- 
gistics problem in establishing workshops that are convenient 
to the teachers. There is also the problem of identifying 
capable instructors who are familiar with PRIMES and who are 
available to conduct workshops. 

File utilization in curriculum activities . The data base 
in the file is extensive. In curriculum decision-making it 
would be necessary to select appropriate data related to the 
needs of a given school district. It may be necessary to 
provide each of the participating school districts with ex- 
tensive staff support to assist them in making effective use 
of the file. This may very well limit the number of school 
districts that can participate in in-depth service. 

Local school district resources . To avail themselves of the 
in-depth service local school districts need to provide the 
necessary time for the teaching staff to participate in a 
training workshop and to attend committee meetings dealing 
with curriculum development and materials selection. The 
school district would have to secure substitute teachers to 
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replace the participants. An alternative is for the 
school district to compensate the teachers for the time 
spent after the completion of the school day or during 
the surnner for this additional work. The school district 
should also provide the funds for instructional costs in 
conducting the workshop. 

Currency of data base . The effectiveness of the file is 
based on its completeness in providing the analysis on 
all basal mathematics programs. It is estimated that 
basal textbook programs are revised at the rate of ap- 
proximately three per year. Consequently, the system 
needs to be updated as new programs are published. This 
requires a complete revision of the indexes, review of 
authority lists, and microfilming the documents file to 
include the new programs. 

Format of indexes . To utilize the system it is necessary 
to be familiar with the authority lists. It is im- 
practical to browse in the file and select items of 
interest. To do this it would be necessary to develop 
another format for the authority lists. One possibility 
is using the "keyword-in-context" format which would pro- 
vide an entry into the system via words that are 
familiar to untrained users. 



ERROR SOURCES 

No systematic attempt has been made to identify the fre- 
quency of errors or the types of errors that occur in the file. 
Proofreading randomly selected microfilmed documents for grades K-2 
suggests that there are two types of errors - chance errors and those 
due to difference in judgment among the analysts. 

Chance errors . These may be due to mistakes made in tran- 
scribing numbers from the original textbooks in which the 
analysts have indicated the code numbers to the summary 
forms. Also, there is a possibility that there are errors 
due to keypunching. 

Errors due to judgment . Discussions with the analysts re- 
vealed that there are a number of items on which there is 
disagreement insofar as coding lessons are concerned. 
PRIMES staff is aware of the need for both a reliability 
and validity study for both of the master authority lists. 
The content analysts have participated in an inter- judge 
reliability study. There are also plans to study the 
validity of the content item assignment utilizing the 
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judgment of the senior mathematics editor as the criterion. 
It is expected that further utilization of the file will 
tend to identify errors. Plans have already been made to 
correct any errors that are found. 



CONCLUSIONS, IMPLICATIONS, AND RECOMMENDATIONS 



The statements in this section are tentative since 
there has been limited time to experiment with the 
utilization of PRIMES. However, the use of PRIMES 
to date has been studied by consultants and ex- 
perienced school administrators. Their recommenda- 
tions are that the system be applied and additional 
data be collected to further evaluate its effective- 
ness. The discussion under CONCLUSIONS deals 
primarily with the development of the file and the 
supporting tools. 

CONCLUSIONS 

The development of the microfilm document file and the ac- 
companying reference tools support these conslusions: 

1. IT IS FEASIBLE TO DEVELOP MASTER AUTHORITY LISTS 
OF CONTENT AND BEHAVIORAL OBJECTIVES FOR ANALYZING 
PUBLISHED CURRICULUM MATERIALS IN ELEMENTARY SCHOOL 
MATHEMATICS. 

2. IT IS FEASIBLE FOR QUALIFIED ANALYSTS, WORKING IN- 
DEPENDENTLY, TO APPLY THE AUTHORITY LISTS TO 
PUBLISHED CURRICULUM MATERIALS. 

3. THE COMPUTER IS AN EFFECTIVE MEDIUM FOR PRINTING 
FORMAT-TYPE PAGES IN CURRICULUM MATERIALS ANALYSIS, 

PRINTING EXTENSIVE INDEXES, AND PROVIDING A METHOD 
FOR UPDATING AUTHORITY LISTS PERIODICALLY. 

4. MICROFILMING ORIGINAL PUBLISHED MATERIALS ON APERTURE 
CARDS AND IDENTIFYING THE CARD CONTENT IS AN EFFICIENT 
METHOD FOR STORING AND RETRIEVING A LARGE VOLUME OF 
PRINTED CURRICULUM MATERIAL. 

5. A STATE EDUCATION AGENCY IS IN A STRATEGIC POSITION TO 
PROVIDE LEADERSHIP IN UTILIZING NEEDED RESOURCES, PERSONNEL, 
AND FACILITIES THAT ARE LOCATED AT DIFFERENT AGENCIES, IN 
DEVELOPMENT AND UTILIZATION OF A CURRICULUM INFORMATION 
SYSTEM. 



IMPLICATIONS 

The major emphasis to date in PRIMES has been using the 
document file as an auxiliary tool in curriculum development activity. 
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The other applications that are suggested in this section would re- 
quire additions to the present system but would be based on the 
developmental work that has already been completed. 



Aid in developing pre-service and in-service courses for 
teachers . Laboratory activities using several of the 
file's components could be incorporated into a course 
for prospective teachers, lift-service programs are most 
effective when related to the adopted text. Using the 
file's content and behavior lists for the series adopted 
would provide the school district with a guide for de- 
veloping an in-service program thatr relates to its basal 
series . 

Aid in individualizing instruction. The file would be a 
useful tool in developing a sequence of both content and 
behaviors to individualize the mathematics program for 
each pupil. It would also identify materials that present 
these concepts in a variety of ways. A pupil who has not 
mastered a concept presented in one way could be directed 
to published materials that presented these ideas in a 
different manner. 

Aid in development of teacher-made tests or selection of 
standardized tests. The behaviors for each of the pub- 
lished programs in the file are an important tool that 
could help the teacher to develop tests that are ap- 
propriate for his class and program. Standardized tests 
can be selected that include the majority of content and 
behaviors for the instructional materials. 

Aid in placement of transfer pupils. The detailed analysis 
of both content and behaviors of existing mathematics pro- 
grams will assist the teacher in identifying both the con- 
cepts that a pupil has been previously taught as well as 
those which are new. The teacher can then develop a program 
that fills the gaps in a pupil's background. 

Assist in a comparative analysis of published programs. 
Through the use of printed indexes, this system will provide 
a tool for comparing published programs. Comparisons can be 
made on the basis of content, behavioral objectives, problems 
or exercises provided for the pupil, and the type of concept 
development recommended. 
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RECOMMENDATIONS 

PRIMES has concerned itself with the elementary school 
mathematics curriculum and has devoted its efforts to analyzing basal 
programs for the elementary grades. The experiences gained in the 
course of this project can be applied to adding new components to the 
information file, extending the project to include grades 7 to 12, 
and applying the procedures with appropriate modifications to other 
curriculum areas. The project has also explored developing a state 
plan for a total curriculum consulting service and utilizing the 
computer to assist in communicating with the local school districts. 

It is suggested that consideration be given to these 
recommendations : 

Adding components to PRIMES file. There is need for ana- 
lyzing and storing information relating to these curriculum 
materials - audio-visual, enrichment materials, and 
standardized tests. There is also need for a comprehensive 
survey of curriculum practices in elementary school mathe- 
matics, analysis of experimental research studies, and 
summaries of important professional literature of interest 
to the classroom teacher. Consideration should be given to 
identifying creative efforts of teachers and encouraging them 
to submit their materials to PRIMES for dissemination. 

Extension of PRIMES to include grades 7-12. There are a 
large number of textbooks and instructional materials that 
are utilized in the teaching of mathematics in the secondary 
school. There is need for developing the analysis tools and 
applying them to these materials to assist the secondary 
school teachers in their curriculum development activities. 

Other curriculum areas. There is great interest in develop- 
ing a system for analyzing curriculum materials in such 
areas as elementary school science, reading, and social 
studies. Developing analytic tools and procedures are neces- 
sary to implement these activities. 

State plan for curriculum services. The Department is in a 
position to coordinate the activities of these state educa- 
tion agencies - the area curriculum centers, county offices, 
state colleges, and state-related universities - in advising 
local school districts in their curriculum development 
activities. The state is not in a position to retain a 
large staff for the purpose of consulting with the approxi- 
mately 500 school districts throughout the Cormonwealth . 
Coordinating the activities of the state-associated education 
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agencies could conceivably provide the resources in 
regional locations that are convenient to the school 
districts requesting curriculum assistance. 

Computer application to curriculum services . Use of the 
display devices in conjunction with the computer is now 
a reality. PRIMES might be able to develop a system that 
would provide for schools accessing a computer directly 
with queries related to the curriculum file. 
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SUMMARY 



"The Development of an Information System for Elementary 
School Mathematics Curriculum Materials: A Feasibility Study" is a 

project concerned with applying the technology of information 
storage and retrieval to curriculum materials that are in use in the 
public schools of the Commonwealth of Pennsylvania. School districts 
are faced periodically with a need to select an elementary mathe- 
matics text series. Few school districts have either the time or the 
resources to develop comprehensive curriculum guides and accordingly 
select instructional materials. The magnitude of the task of examin- 
ing published textbook materials is illustrated by the fact that there 
are approximately 30,000 pages of published pages that comprise the 
basal programs for elementary school mathematics. 

This project is based on the assumption that much of the 
basic analytical work can be done centrally by a state education 
agency and the results of these efforts can then be disseminated to 
interested school districts. Another principle that has influenced 
the development of the system is that the tools will be made available 
to local school personnel to assist in curriculum decision-making. 

The system is not intended to recommend specific basal programs for use. 

The system proposed to accomplish three major objectives: 

1. Developing a system for analyzing, storing, and retriev- 
ing information related to elementary school mathematics 
curriculum materials. 

2. Developing procedures for utilizing the system for cur- 
riculum decision-making by local school districts. 

3. Evaluating the effectiveness of the system in accomplish- 
ing its purpose. 

y Seme applications of the materials developed by the system are con- 
structing curriculum guides, selection of a basal textbook series, and 
selection of appropriate supplementary materials. 

This project was concerned with these objectives - developing . 
a document file and using the file in working with local school districts 
in their curriculum development activities. 

D ocument file . A system document consists of three basic 
elements - the analysis sheet, the teacher manual page, and 
the pupil text page. Each page of twenty published ele- 
mentary mathematics basal series has been analyzed using 
these categories - content, expected pupil behavior, type of 
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problem, vocabulary and symbolism, grade level, pre-text 
activity, and post- text activity. 

t*' 

Assignment of content to each of the lessons was based 
on the Content Authority List which is a master list of 
about 300 mathematics concepts and skills developed by a 
ccmrnittee of nationally recognized mathematics educators. 
The behavioral objectives were assigned from a list of 
approximately 1500 pupil objectives that was developed 
by members of the staff of the Learning Research and De- 
velopment Center, University of Pittsburgh. Content 
analysts were professors of mathematics at colleges or 
universities in the Commonwealth. Seven experienced 
classroom teachers were trained and supervised by the 
authors of the Behavior Authority List in applying this 
list to the published lessons. 

Another component of the document file is the features 
that describe a given program. Each of the features 
described can be documented with a page reference. 



Servicing school districts. To service local districts, 
the project has cooperated with other educational agencies 
in organizing two regional centers. These centers are 
responsible for providing local school districts in their 
geographic area with consultative services following pro- 
cedures developed by the central system. Each of the 
centers is staffed with an elementary mathematics adviser 
and supporting clerical personnel and are supplied with 
the document file, indexes, original basal series, and the 
authority lists. Periodically, evaluation conferences are 
conducted to identify problems and to suggest solutions to 
increase the efficiency and effectiveness of the services 
to the local school districts. 



The results of this project can be described under products, 
services, and application of the PRIMES materials. 

Products 

Microfilm File. The microfilm file consists of approxi- 
mately 25,000 aperture cards. Each card has an accession 
number identification and the grade level at which the lesson 
is taught. 
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Content Authority List. This list consists of approxi- 
mately 300 mathematics concepts and skills that may be 
taught from kindergarten through grade six. The items 
are arranged in hierarchical order with a unique number 
assigned to each of the items. 

Behavior Authority List. This list consists of approxi- 
mately 1500 objectives for elementary school mathematics 
stated in behavioral terms. 

The system has carpi led two master indexes - the Master 
Context Index and the Master Behavior Index . The indexes 
make possible identification of relevant accession 
numbers according to requested content or behavioral ob- 
jective . 

Vocabulary and Symbolism Lists . The system has compiled 
a draft of the technical vocabulary and symbolism which 
are used in the published programs. In addition, there 
is a vocabulary and symbolism summary for each of the 
basal textbook series. 

Features . Each of the books is described in an objective 
style and is documented with a page reference. These 
features are based on an outline which includes such 
characteristics as organization, scope, sequence, and the 
development of the mathematics content. 

Consultative Services - Regional Centers. There are now two 
regional centers established to use PRIMES materials in working 
with local school districts. Three levels of consultative cur- 
riculum services are offered to interested school districts. 

These are: 

In-depth Service. This refers to a comprehensive study of 
the school's current elementary mathematics curriculum, 
constructing a new curriculum guide, and selecting appro- 
priate instructional materials to implement this guide. 

Modified Service. This involves a comprehensive analysis 
of a school's present mathematics curriculum. 

General Service. This includes answering individual questions, 
advising about in-service programs, and identifying competent 
personnel to assist in activities to improve the mathematics 
curriculum. 
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Application of PRIMES in Training School Personnel . PRIMES 
personnel have participated in the training of elementary 
school teachers and students who plan to enter the teaching 
profession. These programs are: 

Workshop for Elementary School Personnel. A two-week 
workshop conducted at the Greensburg Center, University 
of Pittsburgh, to train teachers and curriculum coordi- 
nators to apply PRIMES in curriculum decision-making. 

Methods of Teaching Elementary Mathematics. Two PRIMES 
analysts have utilized the system's materials in under- 
graduate courses. 

Conference for Secondary Teachers on Elementary Mathematics. 
This conference was conducted at Indiana University of 
Pennsylvania to train mathematics teachers to serve as 
coordinators of mathematics teaching in the elementary 
schools . 

Institute for Instructors of In-service Education in 
Mathematics. One-week institutes were conducted at three 
of the state colleges for secondary and elementary teachers 
who have had recent college courses in mathematics. 



The development of the microfilm document file and the ac- 
companying reference tools supports these con clusions: (1) a state 

education agency is in a strategic position to coordinate the analysis 
of a large volume of curriculum materials. It can utilize diverse 
talents at key agencies to develop the appropriate analytic tools, to 
implement the materials analysis, and to produce a document file; 

(2) it is feasible to apply the techniques of information storage and 
retrieval to educational curriculum materials and develop a system 
that is flexible in responding to local school district requirements, 
and (3) servicing local school districts should be done on a decentral- 
ized basis. 

Developing a comprehensive curriculum materials data base 
has implications for other requirements in building a total curriculum. 
The curriculum materials can be related to in-service workshops for 
teachers, individualizing instruction, developing valid teacher-made 
tests, selecting appropriate standardized tests, and assisting schools 
in placing and teaching students who transfer from other districts. 



PRIMES has concerned itself with developing an information 
system in the area of elementary school mathematics curriculum materials. 
The effectiveness of this system supports the following reco mmendations : 
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Adding components to PRIMES file . Consideration should 
be given to developing techniques for storing relevant 
information with regard to such materials as audio- 
visual, enrichment, and standardized tests. 

Extending of PRIMES to include grades 7-12. School 
districts have expressed interest in securing informa- 
tion about materials that would assist them in designing 
a comprehensive mathematics curriculum for grades kinder- 
garten through grade 12. It is suggested that study be 
made of PRIMES procedures so that the system would in- 
clude secondary curriculum materials. 

Developing information systems for other curriculum areas . 
There is need in other curriculum areas for developing 
analytic tools, applying these to curriculum materials, 
and systematically storing and retrieving the information. 

Developing plans for state-wide curriculum services. 

School districts would be receptive to a state-wide plan 
that would provide them with expert services in dealing 
with their curriculum problems. These services should be 
based on a comprehensive data base and be responsive to 
local needs. The state could utilize the existing state- 
related educational agencies in implementing this recom- 
mendation. 

Applying computer technology to curriculum services. 

Methods for utilizing the computer to deal with the 
"information explosion" in published curriculum materials 
should be explored. This includes developing the components 
of an information system and providing ready access to the 
information for local school districts. 



50 



o 

-ERIC 



REFERENCES 



1. Asher, J. William. "Information Retrieval Research at the 

University of Pittsburgh" in The Automation of School 
Information Systems. Don D. Bushnell (ed. ) , Washington, 

D. C., Department of Audiovisual Instruction, National 
Education Association, 1964, pp. 60-66. 

2. Barhydt, Gordon C. Western Reserve University Computer 

Index of Educational Research . Cleveland: Western 

Reserve University, 1964. 

3. Center for Documentation and Communication Research. 

A Selected Bibliography of Documentation and Information 
Research? Cleveland: Western Reserve University, 1964. 

4. Christian, William C. "Retrieval System Disseminates 

Medical Information," Systems. 7, November 1966, 
pp. 16-17, 54. 

5. The . Knowledge Availability Systems Center. Space and Technology 

Transfer . Pittsburgh: University of Pittsburgh, n.d. 



51 



0 



APPENDIX A 



\ 



PERSONNEL 

Project Staff - Department of Public Instruction 
Emanuel Berger, Director 
Doris E. Creswell, Co-director 
Esther Schneier, Mathematics Education Adviser 
Kenneth E. Calkin, Mathematics Education Adviser 
Margaret W. Sharp, Secretary/Reference Clerk 
Michele A. Senich, Secretary/Reference Clerk 

Project Staff - Region G 

Frank Ziaukas, Project Director, ESEA-Title III 

Eugene McKelvey, Superintendent, Westmoreland County 

Gerald Orr, Assistant Superintendent, Westmoreland County 

Edith Kohut, Mathematics Education Adviser 

Mildred Reigh, Mathematics Education Adviser 

Leslie Pfeifer, Secretary/Reference Clerk 

Nancy Ippoliti, Secretary /Reference Clerk 

Project Staff - Region J 

Francis J. Pilecki, Director, ESEA-Title III 
Ruth Sperber, Mathematics Education Adviser 
Joseph Skok, Coordinator, Curriculum Enrichment 
Shirley Jimenez, Secretary/Reference Clerk 



A-l 



o 



APPENDIX A (CONTINUED) 



Analysts 

Charles Barnhart, Behavior Analyst, Baldwin-Whitehall School District 

Annora Deep, Behavior Analyst, Baldwin-Whitehall School District 

Donald Deep, Supervisor of Behavior Analysts, Learning Research and 
Development Center, University of Pittsburgh 

Lois Emler, Behavior Analyst, Pittsburgh City School District 

Willard Hennemann, Jr., Content Analyst, Indiana University of 
Pennsylvania 

Robert Hightower, Content and Behavior Analyst, Philadelphia City 
School District 

Mary Ignatius, Behavior Analyst, Baldwin-Whitehall School District 

Edith Kohut, Supervisor of Behavior Analysts, Learning Research and 
Development Center, University of Pittsburgh 

Kathryn Lanz, ProblemrType Analyst, Central Dauphin School District 

Richard Lundvall, Problem-Type Analyst, Camp Hill School District 

Virginia Martin, Behavior Analyst, Peters Township School District 

Michael Montemuro , Content Analyst, West Chester State College 

Mildred Reigh, Content Analyst, Indiana University of Pennsylvania 

Elsie Reis, Behavior Analyst, Pittsburgh City School District 

Lena Reno, Behavior Analyst, Phoenix, Arizona 

Donald Sapko, Content Analyst, California State College 



Consultants 

Gordon C. Barhydt, Western Reserve University 
Andrew Bellano, Auerbach Corporation 
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Consultants (continued) 

Lawrence Berul, Auerbach Corporation 

Lee Boyer (retired) , Millersville State College 

Thomas Deahl, Auerbach Corporation 

Frederick Goodman, University of Michigan 

Joseph Lipson, Nova University 

Joy E. Mahachek (retired) , Indiana University of Pennsylvania 

Earle Myers, University of Pittsburgh 

Herbert Olander, University of Pittsburgh 

C. Alan Riedesel, The Pennsylvania State University 

Sara Rhue, Temple University 

J. Fred Weaver, University of Wisconsin 



Administrative Staff - Department of Public Instruction 
A. G. Breidenstine, Deputy Superintendent 

Neal V. Musmanno, Assistant Commissioner, Office of Basic Education 

Robert B. Hayes, Director, Bureau of Research Administration and 

Coordination 

John E. Kbsoloski, Director, Bureau of General and Academic Education 

Donald M. Carroll, Jr., Director, Bureau of Curriculum Development 

and School Evaluation 

Carl E. Heilman, Coordinator and Mathematics Adviser, Bureau of 

General and Academic Education 
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Adininistrative Staff - Department of Public Instruction (continued) 

Richard A. Gibboney, (formerly) Deputy Superintendent of Research 
and Development 

N. Sydney Archer, (formerly) Director, Bureau of Research 

Stanley N. Miller, (formerly) Director, Bureau of c^heral and 
Academic Education 
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BASAL TEXTBOOK PROGRAMS IN FILE 



Publishers 



1. Addison-Wesley Publishing Co., Inc. 

Elementary School Mathematics 
Eicholz, Daffer, Brumfiel, 

Shanks - 1963 

2. American Book Company 

Elementary mathematics series 
Deans, Kane, Oesterle - 1965 

3. Encyclopaedia Britannica Educational Corporation 

Math Workshop 

Wirtz, Botel, Beberman, Sawyer - 1964 

4. Ginn and Company 

Mathematics We Need 
Brownell, Weaver - 1965 

5. Harcourt, Brace and World, Inc. 

Elementary Mathematics 

Clark, Beatty, Payne, Spooner - 1965 

6. D. C. Heath and Company 

New Ways in Numbers 
Elwell, Sister M. Stanislas, 

Fitzgerald - 1965 

7. Holt, Rinehart and Winston, Inc. 

Moving Ahead in Arithmetic 

Merton, Brueckner, Grossnickle - 1963 

8. Holt, Rinehart and Winston, Inc. 

Elementary Mathematics; 

Patterns and Structure 

Nichols, Flournoy, Kalin, Simon - 1966 

9 . Houghton-Mif f lin Company 

Modem School Mathematics; 

Structure and Use 

Duncan, Capps, Quart, Dolciani, 

Zweng - 1967 
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Grades 

K-6 

1-6 

K-6 

1-6 

K-6 

1-6 

K-6 

1-6 

K-6 
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10. Laidlaw Brothers K-6 

Sets, Numbers, Numerals 
Gundlach, Welch, Buffie - 1965 

11. McCormick, Mathers Publishing Company K-6 

The New Arithmetic 
Weber, Weber - 1963 

12. McGraw-Hill Company (Webster Division) K-6 

gjgjgttgB i: M athematics; Conce pts, 

Properties, and Operations 

Spitzer, Banks, Bums, Kohrs, Folsom - 1967 

13. Charles E. Merrill Books, Inc. 1-6 

Discovering Mathematics 
DeVault, Osborn, Swenson - 1964 

14. William H. Sadlier, Inc. 1-6 

Contemporary Progress in Mathematics 
Bezuszka, McDonnell - 1965 

15. Science Research Associates K-6 



Greater Cleveland Mathematics Program 
Education Research Council of 
Greater Cleveland - 1962 



16. Scott, Foresman and Company 1-3 

Seeing Through Arithmetic 
Hartung, VanEngen, Gibb, Stochl, 

Knowles, Walch - 1965 

17. Silver-Burdett Company K-6 



Morton, Gray, Rosskopf - 1966 

18. L. W. Singer Company K-6 

Sets and Numbers 
Suppes - 1965 

19. School Mathematics Study Group K-6 



Panel on Elementary School Mathematics - 1965 

20. Xerox Educational Division 1-3 

Mathematics laboratory Materials 
Rasmussen, Hightower, Rasmussen - 1964 
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FILE PARAMETERS 



DOCUMENT FILE (Kindergarten to Grade 6) 

Number of programs 20 
Number of books 131 
Number of pages/book 300 
Total number of pages (Kindergarten to Grade 6) 39,300 
Number of aperture cards (200/book) 26,200 
Number of 35-itm rolls 65 
Cost per duplicate 35-rrcn set ($6.00/roll) $ 390 
Cost per duplicate aperture card set $1,500 
Estimated number of 18x24 prints - Harrisburg 20,000 



Estimated number of 18x24 prints - regional center 10,000 



BEHAVIOR AND CONTENT AUTHORITY LISTS 

Number of categories 1,800 

Number of pages/listing 100 

Frequency of updating 4 

Levels of indention 7 

INDEX FILE 

Average number of index entries/lesson 8 

Total number of index entries 320,000 

Number of pages/book-form index 4,000 

(40 entries/column on a two-column page) 

Cumulative supplements 4 

Number of pages/supplement 2,400 
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